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ABSTRACT ^ vv . 

several aspects of interview research heretofore 1 
receiving little attention are discussed>, A brief description of the 
differei)it types of ih.terview formats and 1^ analysis is^ 

presented. Pbllpifing a discussion of th's^proDiem^olf analyzing 
protocol da^a, some suggestions are of ferfed about analysis procedures 
that derive from con^structionist assumtions. \ model is offered of 
the interview which describes its role in hypothesis formulation and 
hypothesis testing. Views on how the interview can be used" in 
combination with other research methdds to investigate problem \ 
solving are discussed. Finally, how interview research is c^urrently 
being reported is examine4V and recommendations concferning the types 
of information necessary for inclusion it^ such reports are offered. 
Sugqestions are aimed at^couraging the. researcher to remain 
skeptical of interpetations of protocol data, and to report the 
results of interview research fitting speci^fic criticism from the 
research community. (Author /GK) 



Reproductions supplied, byi EDRS are the best that* cjan be made 

from*the original document.' 



«««««««««««««««« 
* 



ERLC 



' r 




Analysis and Reporting* of Interview Data 



Clifford E. Konold and Arnold D, Well 

. . , - ■ >v - . ■ /.■■:■■ 

Cognitive Development Project' : 
Department of Psychology 
University of Maissachii^etts, Amherst^ 



UA^KPMinilENt OF EDUCATION 

NATIONAL INSTmrre^ EDUCATION 

EOUCAJIONAL RESOUBCfS INFORMATION ." 

CENTER lERiC). j! 
yr This document has b«en reproducad as 
roceived from the^person or. orgafuzation \ . 
onginatir>g it. . * * 

.D Minor char>ge%havq bean made to improva 
reproduction qualiiy. 

•■ Poihts of view or opiniorvi stated in this docu- 
ment do not nece^riiy represent. offtcial NIE 
positioirt pr policy/ . '^jg^ 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN QRANTED BY 



JO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)/* 



y 



), ■ 



<4 



Paper prepared fpr presentation at the Annual Meeting' of the American Educational 
Research Association; Los Angeles, CA, 1981. This paper was supported in part by 
NSF Aw$rd No. SED78-22043. ° ' ; 



ERIC ' 



2 



^ ■/■^ Analysis and Reporting df Interview Dat^ • ' - 

■■(. '^■^-- r.^.- "'^ 

For the major parjt of this century, American experimental psycholbgists 
interested in the study of learning and ' cognition have employed methodologies 
Xthat require subjects to respond in limited Ways to carefully structured situ- ■ 
ationsi inspired largely by the work of Piaget, however, there has recently ' ' 
been a feawake^x^ to the advantages of considering as data the verbal reports..; 



of subjects l|ngaj^Sd in' a complex cognitive ta§k. I^ problem-solving researchf, 
for example', verbal reports not" only provide information- about the end result ^ 
of the probl^ solvxng Jir^ess (th^ answer) , but are also a rich source of data 
.. concerning the reas'onil^g which led to the final answer. Having some insight / ^ 
into the processes involved in solving a problem not only is imporjtant in -the 
^formulation of models of human cognition but has important and obvious .pedagogi- ,. 

• ■ ■ /• ' . L ■ ■ ' ■ . ■ ' A 

cal implications. ' 

In this paper we will discuss siaveral aspects of interview research that 

' ' . - ■ . ■ • ' . ' 

have received little attention. After presenting a brief descriptio|x of 
different types of inter^ey formats ^^d levels of a^nalysis, we. will discuss 
the. problem of analysing w^tocol dati> and offer ,s suggestions^ about ^ 
s analysis procedures that deMve from constlructiyist assiimptioas. Then we will 
present a model of the intenVew which describes its role not only in hypothesis"* , 
formulation, but also in hypotntsiS testing, and discuss bur^views on how the Vr*^ 
interview can. profitably be used \n coinbinatio^ with^ther research methods to 



\ 



investigate problem solving. FiriaAy, we will examine how interview research/ 

is currently being reported and of f e\ some recommendations concerning the types 

■ • ' ''^^ ' - . - ^ .... 

of information that are -important to mclude in such reports. A miajor point 

whic)i i^ili be emphasized throughout .theVg^per is the importance of taking^^ 

critical orientation .toward hyp.o theses derived from olir i^search, that >the 



lack of ^ such an orientationo5tands as the iiiost^ serious threat to the. aceeptance 
pf the interview a^ a legi'tiinate method pf scientific, inquiry. Pur suggestion^ 
are aimed at encouraging the researcker' to remain skeptical of interpreta- 
tions of- protocof -data, and to report the results of interview research in ,a * ? 
way that/ will l^ermit specifiq criticism frpip the research conjnunity. 

? * ; ^ .' 'laterView Terminology *. ' " \ . \ 



• ■ • / • - ; ■ _ : ■ . •« ■ 

/ There does not yfit^'exist a wid&ty accepted terminology to describe^.the 

various types of research interviews; At. one extreme, the term "interview," 

as used in the Social psychological literature (e.g», Cannell and Kahn, 1968), ' 

can ref^ to a verba*! response to a. single question "^uch as "How old are you?" 

At the Other- extreme, the term can be used to/dascribe an unstructured and coin- 

plex exchange between ;a-' student and teacher about a difficult problem encountered 

in the classroom (Davis, 1975a , 19.75b) • Similarly , analysis of inter'view proto-. 

cols/xjn involve the coding of whether or, not arcorrect solution was obtained, or 

an extensive treatment of every statement made by the ; subjects. Though there are 

numerous ways in which* an' interview, could proceed ind be analyzed, we will rafer 

to three types of- problem-solving interviews and three levels of .protocol analy- 

^sis. The^erminology we employ is meant to 'capture only gross distinctions, but 

wfll serve the purposes of this papef- 

Interview Formats .. ^ . 

In terms of the format of the interview, we will refer to three general 
styles. These differ primarily in the types of follow-up questions (probes) 
that ate permitted: 



' Thinkiiig>?tfl6ud igt-eWiew / In the tH 
giv^ a problem and 'asted to verbalize 
it. They are instructed not to engage 
previ6us thoughts ^; Probing, if used at ^M/ 
aging\subjects« to. vocalize more (cf . , Ericsson 




ud interview,,; subjects are 
hts as they attempt to solve 
is or to reflect 'back* on 



rally restricted \ to encour.-- 



imon, 1^0). 



Indepth intervi^ . Also; referred to\as th^PMgetian *or clinical interview 

^his method also; involves presenting subjects^ v?^gi a specific problem and th^en. • 

questioning th6m as they attempt to solve it /^^^fcppobing technique, however^ 

is much more flexible than with the thinking-aloiiC instructions . Subjects are 

often asked to reflect bacK on -what -they have just done and sometimes offered 

subtle .challenges to their thinking. The interviewer, however, neve if purpose- 
» 

fully gives evaluative responses, nor provides hints in the form of questions, 
statements, gestures, etc. (cf. Fredette, 1979). 

* Tutorial interview ^ In the tutorial interview, the interviewer is inter- 

ested in elipiting a cc^rrect ^blu^ion, but generally tries to provide^ no mare 

*• > ■• ' ■ . • . \ ' . ' ■. 

help than is neces^ry. Probes are therefore permjLtted which lead subjects 

toward solution strategies that may"Sfever haye pccirred to them. 



Level^^of Analysis 



• ^We Will refer to three generaf levels of p^tocol analysis: 

Coded analysis . Coding involves identifying key elements of interest in 
the protocols and defining them in such a way that raters can go through the 
protocols noting the presence- or absence of the elements. These might include 
the use of key words, phrases, equations, problem-solving strategies as Well as 
gross to fine motor responses. , / 



. >y ; ' 



. Descriptive analysis / With thi& method; the reseaxGher is usually inter- 
ested* in providing a clear r^statem^t of:^hiat the subjects^staid and did during 
the i^apview. The attempt is made to describe" the data as they are, making , no , 
inf)^ence about underlying stru'ctar^s' that may account €or the data. The.fotrus 
is on the surface strticf.U3fe^ o^^ subje<^ts' verbaiiaations, on the meanings they ^ 
areN,^xplitit.ly trying tQ cominuaicate. / ' 

Interpretitj-ve Analyg-is , In interpretive analysis, inferences are made 

•'. 9 ■ ^ -^f ^ ' . . ' ^ ' ^ ' 

about the deep ^tructuic'egi of subjecjts? reasoning processes; The researcher use.§ 
what the subject says and do^s to make statements ^about the process and knowledge 
structures (both explicit aO(J tacit) the sjibjects arte using to solve the problem. % 

In Table 1, we have categorized nine 'problem-solving fj^dies with respect 
to the method of interviewing and analysis procedure employed. This should not 
be conisidered a representative sample. The a^rticles were obtained either from 
oiir files or the Journal of ChiWren^ Matbematixal Behavior . .Though we will 
refer to these studies in mote detail in the final^section, jwe include them 
Heije^o illustrate our tejrmihoXogy. 



Insert Table l about here 



\ 



Referring to these reports, one can appreciate the extent to which inter- 
viewing styles and analytical methods vary from study ito study and, thus, fee ^ 
difficulty in categorizing tticm as Ve have. A single interview can proceed 

\ - \. y . ' ^ • 

through different phase's. It can b^g^n.^with the intervii^wer liising only facili- 
tory pro^)e5, if needed, cliaracteristic of the thinking-aloud interview. 'Once a 
solution t^s been arrived at^/nore indepth probing may bejiised, followed, if ^ , 
required, by probes designed to lead the sub'ject^to a correct solution. We ^ 

would, however, classify this tatorial interview, though the investigators 

' •■' ••.■•■■)■. , ■ ■ ■ .. ■ , ■ , • • . ■ 
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may (and should) regard each phase of the interview differently (e.-g.^ Rosnick . 
and Clement^ in press) . Even more diffitult to classify are the levels of analy- 
sis • Many studies which seem to be interested in descriptive or interpretive 
analysis also'code parts of the interview, and it is' often the case that 
entirely coded protocols are used to support descriptive or interpretive ^jcalje- 
. ments, If;-cjb4ed information was included in a ^report, yet ^the attempt wa^ made 
to accountVfor all or most of the subjects' verbalizations and behavior, (it was 
categorized as a ^descriptive or interpretive analysis. Therse studied 
include large segments of protocols along with their analys'is (e.g., 
and Clement, in press). If only selected portions of the ii^terive^ wer^ c^ded, 
it was categorj.:zed as a coded analysis^ ' 

,To ilXustrate the distinction between descriptive and ^terpretiv^ aj(|alysis, 
we have inducted, a segment and a brief analysis of a tutorial interviejw donducted 

on re( 



>0 whi 



by John Clement: (The transcript of the entire intervjLew is availabl 
The student being interviewed (Mark) was asked to construct an equat 
described the relation between vejlocity, acceleration and time for a 
off a eliff . Each vetbalization of the interviewer (I) and subject /(S) was 
numbered consecutively- from the beginning of the interview. ' 



quest. ) 
ch 

ball dropped 



59 I) Suppose the velocity is . ; turns out to be tv^ipe the time /everywhere . 

As. "t" goes f r<>m I to 2 to 3, velocity goes ,f rom 2 to 4 to/ 6. 

60 S) Uh hub. .-. 

61 I) ^at, would an equation be for that? 



62 S) The velocity ds always twiire as large as the time. 

63 I) Right. ' 



64 S) So. . 2. "v" equals "t", I asl^iume. 



65 I) Ok. Write that doyn; . * * - • . \ J 

66/S)\ (Writes "2v=t") Ok. . ahl;^ ; * 

• - • • . ■ • . . ^ . .* •. ' ■ , . 

67 I) Now check that out and $ee if it works. . ■ 

• - • . . • . . ••■>••.■ - ■ , 

78 S) Ok. «t" is 1, then "v" is '2. If "t" is*2, th'en "v" is 4, or you could 

• actually think of this as 2 times Vt" equals "t". / , ^ 

791) Ho* does -that work? ' ^ . v 

0 ^ 

you <cduid replace this by ''t'*, and you could ;^ay 2 times "t" equals "t", 



80 S* ■ You. se^th^4,uai:. Sine, these two are: e^ual; -thi-s is the sa.e..nd 
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In line 62, it seems as if Mark has comprehended the nature^ of the relation 
he is to express, in that he provides a correct restatement- "But the: eqii ^tiojw 
he suggests ine 6^ is the reverse of the correct equation^ 2t=v. It is evi- 

dent from the remainder, of the interview that he did not simply make a careless 

\'' ■ ' ^ .1 ■ , . ■ . ■ ■ •■ ■ .■' ■■ ■ ^ ■ V ' ■ - ■ 

* • * * ■ J, ■ 

mistake. It therefore seems- clear j in a descriptive sense, what he has communi-^ 

^ " • " ■ ' ■ ■ . ■■■ • ^ 

cated^ But why does he think that "2v=t" is the appropriate equation?' This he 

■ 71 , •. . - ■ ■ ti 

doeisviot explain^-it just seemed to him to be correct. ' \.: 

J Lines 78 ^and 8b demonstrate why a descriptive analysis is not always 

■ . • • * 

'straigtitforward. As ,a result of plugging numbers into the equatioo "2v=t'*', 

,. ■. *■ ••• ' \ » ^ ■ « 

-.. K ■ . . ■ \ y ^ ' . ^ ■ \ ^ ■ ' 

)Mark proposes that an equivalent equation is "-2t=t". How he arrived at this 

con'cluision is 1^01? obvious. But we thiiik what he did was*^ to replace^ first "t" 

■ . ' * ■ . » • ^ •*..'''•'■ 

and then "v" with 1, 'multiply the ^left-hand side (LHS) by 2 and thus bbtaifl the 

y'^'r .-.■^ 'V • ' ---^ . • ■ ' ' ■ ' ■ *. ' ^ 

solution 2kX* Sin'ce the LHS which has been associated with "v" is twice as 
lar^e as , the;MIS ^hick ha with "t", the relajtiori, in his mind/ 

has been 'accurately expressed. Replacing the variables with 2 ^hen 4 also 
maintains the^ deisired relation arid silpports iiis feeling that the. equation^ as ' 



written, is correct. Mark then correctly reasons that since he is replacing 
both variables with the s^^ values , he can use identical variable symbols on 
both sides. This descriptive anal^^s is an hypothesis about wha^.>the subjecJt" ^ 

is trying to communicate. Givien that>the hypothesis is accurate , we s,till do 

■4 - ' ..... . ♦ 

r ■ ■ ■ 

not know why he is <foing what he is. An interpretive.analysis 'would hypothesize 
what knowledge Mark possesses that permits vhim to view. equations as he does, 
that* allows one side of an equation to be twice as large as the other,' (We * 
will suggest^an interpretive hypothesis concerning Mark's solution later in the^ 
paper.) ^ ^\ * ^ . 



Protocol Analysis 



Having distinguished among types of interviews and^ analyses , we are -now 
prepared to address what we^ consider 'the most, difficult aspect of interview 
rdsearch-r-analyzing the data. The most straightforward analytic approach is 

to code selected parts of the interview, transforming them into more manage- 

t ■ ^ ■ ■ ■ 

able, quantitative data. A methodology -for coding is wej.^ established, and the 

resultant quantitative data can,, if desired, be further subjected to standard. 

Statistical te&t§. If descriptive or interprietive. analyses are desired, how-^ \ 

ever, researchers are virtually left to their own. Little has been written 



concerning a method for such analyses J In this section we will attempt, if hot 

* ' * '• 

to establish more formal procedures for descriptive and interpretive analyses, 

■■ ■ ■ ■■• ■ ' 1/ ■ ^. ' - 

t|ien at least to encourage more dialogue about how a .researcher might, "make up 

'i ■ , '. . . . / 

^jor the uncertainties TlNt^ie method of ^interrogation by sharpening the subtJleties 
of his-intetpretation." (Piaget, '^1929, p. 9) 
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A, Gonstructivistv Vi^w 



♦ We begin .by considering |the problem of apalyzing protocols within a* coii- 
structivist framework iand by locking at the interview from the point of. view of . 
the subject, the researcher, and the research community. . The fexplicati9n;bf 
xronstructivist assumptions is relevant because we, as most xesearchers using 
---^iliterviews to study problem solv^^, have been influenced by Piaget and offer 
constructivist interpretations of learning aad problem solving^^^^e feel that' . 
-these same assuniptioas should inform our methods^ and consequently our sugges- 
tions concerning protocol analysis derived from them. 

; . According to a constructivist view, information or knowledge is not avail- 
able in raw form to be picked up by passive human receptors. Rather, perception 
is. a selective process, insofar as celrtain features of a stimulus, are noticed 
and others ate ignored. Jt is a Constructive procees since even those aspects 
which are selected from the environment are not simply, in the process xOf 
. selection, made available for human cognition. Rather, perceptions are con- - 
structions and, therefore, their •nature is djE^termined at/least as much by tlje 'N^ 
workings of the perceiver as it,i§. by their presumed independent existence j * 

in the external environment*. * ^ * 

■ ■ , \ ' ' % ' 

•" ■ '■ *^ j; • ■ ■ ' 
Vhen suDjects are' presented yith a stimulus in the form, ^y, of a 

statistics 'problem, they attempt to assimilate dr^make sense of the pr<^ble\K^ 

in terms of existing knbwledgd^. Existing cognitive structures permit the ^ • 

■ ideiitif ication of the nature of th^ problem and the relevant informatioi>, and * 

the specification of a pl^n to arrive at the solution. Such information is 

not inherent in the problem but is selected and simultaneous*!l^y given -particular ^ 

•* ' ' ' ' ' 

meanings via existing knowledge. . . 

■ ' ■ . ^ '"'^ ' 

Psychological research, by its* very nature,> is reflexive (Gadlin and 

• . ■ ^ v ■ • - ^ 

ingle, 1975). That^is, if we hold particular views about how ahd why humans 

- . . -f. • . . . ^ 



er|c • ■ 



bel^ve as they do, and i^f we Jtake these views rS^xiously, they ought to be equally 



r 



powerful in describing the behavior of psychologists* Recognition that as 



researchers of human phenomena, we' are both the subjects and objects of study - ' 
Srings us face to face with the contradiction between^ the objectivity required 
of a scientist and ^he subj[ective nature of human existence. As we attempt to / 
analyze, protocols, howeyer, .^e feel it is- critical that we recognize the limits 
of our objectivity — that we take into account the subjective nature of our 
taisic. If we assume that iii solving a problem, subjects 'a^r.e guided and limited 
by their existing knowledge, we must also be ^w^e that, dur prior knowledge 
plays a similar role as we attempt to analyze the r'esultan't^^^d^ta. . The 
-nature of our task as researchers is almost identical to that of our subjects', / 

differing only with respect to the particular form of the problem we confront-- 

- ■ • * ■ • • . • * • • .' . - 

^■ - " ■ ■ . • ' 

bur. subjects must make sense of the problem we give thenf; we must make sense 

of their verbal reports in the context of the problem. The same dynamic is 

opetative wheji, colleagues read *aa article in which we, have reported the v, 

results of' our research. TJieir prior knowledge serves as the means by -which [• 

they come to an understanding of our- interpretation of the subjects* under-^ 

standing. Indeed, they can often learn more fz^om a report^ aDojut* the way we, 

as researchers think than they can about how. subjects solvetf^he problem. 

' ■ ■ * ■ ■' ■ ' . • * ■ ,'' ■ , 

To say that human perception is totally . determined by prior knowledge, 

.«•'■«.' ^ ■, ^ • 

Abwever,- is 'incorrect . Information is never totally assimilated*-some' accommoda- 
tion always takes i);lace. Our objective as researchers is, in fact, to accommo-i 
date to the. protocols we collect — to have them inform us rather thdn simply con- ^ 
firm prior expectations. The comments we make below are intended to encourage 
accommodation during the process of protocol analysis. 



. and Suggestion? 

Research is undertaken with some general goals and assiiraptions that deter- \, 
mine not only the questions Ihsked ^^nd the methods used but also the way in which. 
dataj S>nce collected, are^^alyzed. To understand the rese2irch and the conclu- 
sions! derived from it, these assumptions' and goals must be implicitly shared by 
the reader, and the reseacher, or they must be explicitly stated as part of ^the 
research report. Our discussion of protocol analysis is based largely on our., 
research experience, and we will attempt, therefore, to make explicit the ^maj or . 

• ■ * " • ■ t , ' ' •■ ' ■ • • ; * • ^ ■ ^ 

\ ■ . ■ ■ ■ k ■ ' • - • 

goals and assumptions which prescribe our methods of analysis. >^ . 

Oiir primary interest has- been to explicate s^tudent understandings of 
statistical concepts that we find difficult to; tea'ch in the classroom. If we * 
can learn more- about the types' of prior knowledge students bring to the class- 
room, we should be ablfe to alter our instructional apprqrach in ways 'that will 
be beneficial to our istudehts. With thiS/ goaL in mind we tend toward intetprel- 
tive as opposed to coded* analyses . We wish to develop tnodels of student j5roblj^m 

.- ■■■ . ■ - ■ . " • ■■ ■ ,v 

solving that are powerful enough to capture important individual differences , yet 
not so specific that wie have as many models as> we have students. During analy- 
sis, we alternate frequently between indepth analysis of individual protocols 
in which we try to undex^stand what a particular subject is. thinking about 'x', 
to a more general analysis A which we ask what characteristics do all subjects,"^ 
or a subgroup of subjects, ^^Jhare in thinking about 'x* . vThis strategy keeps us 
at a level^ of analysis that a].16ws us to generalize our understandings beyond 
the individual, yet does no«t result in* information that is too global to be of 

any value in instruction. It also provides us with an interactive^ framework 
V ' ■ " ■ " ■ . 

for hypothesis formulation and' testing. Hypotheses formulated on the basis of 

ii single. protocol are ''tested** on other protocols; hypotheses about group per- 

formance are "tested- on individual protocols. In this way we^ope to avoid 



the. extremes of either nomothetic or idiographic approaches. ^ ^ 
In analyzing individual protocols, we attempt to,, construct a model that 
' can account for statements by a subject in ^uch a way that no statement seems 

vcpntrradictory or incompatible with, another. The assumption that all of a sub- 

' ^ ■ ■ 

^ject^s statements are logically 'cpnsistent is *dei?tainly unfounded. If we do 
not jmak^ it, however, it is too easy to discount Sections of a protocol 
that a r^ inc^sistent with seemingly understood sections by attributing them 
to unnoticed verbal slips or »to subjects" inability to verbalize what they are 
actually dding. Plaget (1929) established criteria for differentially con- 
sidi^ring statements 'made by . children, j He felt, for example, that some state- 

I ■ ■ . . ■* ■.■.»■•■', * -. I 

ments are playfully uttered arid 'are not intended to be-taken Nliterally. With 



• college ^students, however'i^ unless^ 'the sJfcject seem^ flippant Or\^^bellious, all 

' • ■• •. . .'■■'^ ■ V ■ ■■ * , • " 

statements should be con^iidered as data^^^^^ for. 

. * The assumption that statements made by subjects are, from their point of 

' ■• 0 ■ * ■ ■. ■ . , ^■ ■ ^ i ' '.■ ^ 

view, logically consis^rit provides us with a criterion for analysis*:* In 

geqeral, hypothesis A should /be considered better, than hypothesis B if it 

aiccounts for more of the subject's statements. This in turn suggests that 

atjtention should be focused on statements that seem somewhat conf usin g, that 

at first might be..'disregarded for the very teason that they do not fit with our 

underlstanding of 'the other, statements . In attempting to take such data into 

account, different hypotheses will be -considered. Frequently we have found it 



to be^hef'^ase^iJiiat jiew explanations.^^ into account pr-eviously ignored 

Information reqtiire that different interpretations be ^iven to those sectiohis ^ 

■• ;\. ■ ■ ; -v'^ .• • ■ ■ i ' - • ■ ■ ' ' 

of the protocol that intiaij^ seemed to be best understood. 

From a constructivist point 'of view, the major problem confronjling .the 

researcher who wishes to ana],yze protocol data is not that 'of generating . 

'/ explanations. We tend to notice^ in. interview protocols those things that 



make sense to us. Guid'ed by Hypotheses held prior to the interview, or by 
those /derived from only a limite4 amount of data, explanations can be quick- 
ly advanced and data which support these^explanations are easy to find. This 
problem underlies all scientific inquiry. (Kuhn, 1962). Indeed, there are no 
methodologies y no .tigorous criteria ot formulae which* will prevent "scientists: 
from usually seeing what they expect to see. As compared .to positivistic experi- 
mental methodologies y an advantage of the interview technique and. an accompanying 
constriK:t.ivist epistemology , is that its adherents might be less inclined to^view 
their me^ods as objectively-paved roads to truth. Carl Jung used to instruct » 
his stud^ts to forget; what they had' learned about symbolism when they analyzed 
a dream.. He Relieved that onie could "never understand s.omebody else' s dream 
well enough to interpret it correctly." He continually reminded himself of this 

V ■ '^ ■ • ' ' " ' ^ ' ■•' ~ : ■ ■ ■ 

"in order to check the flow of . ..associations and reactions, which might other- 
wise prevail over my 'patient* s uncertainties and hesitations. " (Jung, 196,4, 
p. 56) A cbnstructivi'st orientation can serve as a needed reihinder that 
inferences drawn from'data are always^ to a greater or lesser degree, invalid. 

Sullivan (1954), in reference to analysis of the psychiatric interview, 
cautioned that early impressions of the interviewer are "rough hypotheses, and,* 
like all hypotheses in interpersonal work, they should be subjected to continu- 
ous, or recurrent, test and correction." (p. 121) This advice is equally 
valid' for analyzing protocol data in problem-solving interviews. Care must be 
taken during -the .early stages of analysis to scrutinize, i^tial explanations 
One, way to do this is to attempt always to consider more than one hypothesis 
(cf. Rubin, 1975). This is a difficult task, however, and one must fight /the, 
tendency to construct straw-man alternative- hypotheses wh^h can setye to 
falsely confirm the superiority "of the favored, initial hypothesis. . 

Something that we have found to be an effective me^s of generating 
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alternative hypotheses is to have the protocols analyzed by a group of inves- 



tigators. While each individual might arrive too quickly 



what in the interviews is important and what understanding the subjects have 



of the problem, these hypotheses tend to be different for 



group. Much of our time in analysis iis taken up by "negotiating" explanations. 
One member will advance an explanation and support it vi^h data. Another member 
will refefr to data that do not seem to support the explanation and might offer 



^t conclusions about 



each meltiiber of> the 



an alternative account. Thus, not only does a group help 



bring * differing points 



of view together, it also increases the working store of data. Working together, 
we are forced to consider data tl^at would likely have been. neglected had each of 
us conducted the analyses on our own. Moreover, group dis;cussions about the 
plausibility of the explanations thus generated often prompt us "to revie)W the 
data. Explanations are subsequently modified, an^ the process continues until 
the group is in general agreement^abqut the adequacy of tie explanations. 

There are some, characteristics of our group, that lenc themselves well to 

, ■ |;-, • : : ■■■■ . 

this process and that are vital to its success .^Perhaps riiost important is that ^ 

while '^iir backgrounds are sufficiently similar that we car. communicate well, 

our outlooks are dif^eteht enough that we often disagree even about major 

goals of our research. Secondly, no member of the group qominates the olihers. 



If this were not the case, some members might not support' 



alternative explanations or criticisms*" TjFirial^l^ the size 
group (3-4 membelrs) is sufficient to ensuife diversity, but 



or even advance 
of the analysis 
not so large as' to 



4 

are 

result in endless discussion. \ J ' 

Once rather stable explanations hav^/^een developed in this smaller group, 
we. present our (Conclusions to memfi^rs of an extended group 



view research in different content areas. The feedback we 
back to the data and the negotiation process. 
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who are doing inter- 
receive can send us 



' Challenging Hypotheses Derived from Prot'ocolg 

■ ; : ■ : / •- : 

"vAs we have portrayed it, the analysis of protocols involves a'cyclical 

process in which hypotheses are develo^fed from data and then tested on the 

• ■*■>■ ' ,' " ' ^ 

same data set; ; Protocols are a rich enough dSta source so that this cyclical 

process can continue for sol^e time before stable explanations are developed. ; 

Once stabilization has>occurred, hypotheses can and ought to be more formally 

tested by collecting mor6 data. In this section we will argue that interview^ 

can be used not only in formulating hypotheses^ but in testing them as well. 

We will aljo advocate the use of interviews in combination with more tradi- /'^ 

tional research methods. ^ 

^ In Figure 1, we havfe attempted to characterize the interview in /relation 

to various types of hypotheses and jjjesearch methods. Rather than viewing the 

inti^rview solely as a means of g;enerating hypotheses, we see it as a stage of 

research in which current hypotheses can also be ^tested and revised. We will . 

elaborate this point in the process of describing Figure 1> 

; Speakiitg of the, effective interview (1^9) noted that "^at every 

moment Jxe must have some working hypothesis, some theory, true or false, which 

he is seeking to check." (p. 9) We fee! that one cannot avo^id such working ' 

hypotheses, that they alwaj^s precede and guide the search for information. In 

the case of the interview ^ these may .be the very informal hypotheses chara.c- . 

■ . >, ' ■'■ ■ . ■ ■ '■' , ■ . ■ ■ ■ 

teristic of new areas of research, or they may be highly developed' thisories 

',' "^^ ■ ■ ■ . " . ,v*' 9 ' 

which have grown out of years of research. On the basis of these hypotheses, 

^- . . ■ . ^ ...s. ■ . V ■ . . t . ■ .. . ^T^ ; ' • 

one or more problenisr aVe cpiistructed. Depending on the spiecificity of the 

current hypotheses, these problems may be unstructured and may even be given 

spontaneously, or they may be ^carefully 'thought out, complete with anticipated 

probes designed to -further elucidaite subject thought processes. ' , ^ 



[nsett f^igure 1 about here % 

] , The simplest form of the interview consists of a problem ,or question and 
the subject's response to it, with ho interventions by the interviewer. A 
modification of this type of interview involves ^he interviewer^ as a facili- 
tator who interjects probes to keep the subject focused on this^, problem and 
riesponding verbally. , This modification is indicated in Figure 1 by dotted 
lines connecting* Probe to Respoiise . In the indepth interview^ probes are 
additionally used to test hypotheises which derive from subject responses "(as 
indicated by the jotted lines connecting Response to Hypothesis ) . Jt is this = 
ability .to reyisie and test hyp^othesies during the process -of data collection 
^'Vhich constitutes the greatest striength o^the^ intelrvie.w^ However, 

t^is -fluid, ongoing exchange between hypotheses and data is the very characteris 
tic of the indepth a.aterview \7hich can make it diffidult to evaluate the valid- 
ity of the "^research. So many hypotheses may hav^e been entertained during this ^ 
interview that it becomes difficult to specify ways in which the data have V 

' ; . • V , ■ ■ ' ^ . ■ • r ■ . ■ ■ ■ 

:'{'.'■ • " . . ■ \ ■ 

been influenced by the interviewer. . . . 

A critical analysis of file interview data should try to take into account 
any effects that hypotheses held by the interviewer mky have had. In addition, 
it '^should; repeatedly test interpretations of the datai by determining the extent 
to which explanations account for the entirety of the statements made by the 
subjects. Arrows going bath ways between Revised Hypothesis and the Interview 
represent this process of formulating, testing, and reformulating hypotheses. 

Thus the testing of hypotheses occurs not only during the analysis phase, 
but also during the indepth interview. While these tests may permit the 
researchers to reject many hypotheses and may allow a degree of flexibility. 



'unavailable with other research techniques, they are not, in general,, the 
types of tests that are convincing to or open to criticism by those not 



directly involved in the research. We feel, Jiowever, that research in problem 
solving is not well'served by the exclusion either of interview studies, on 
the grounds that they are sjubjective analyses of introspective data, or of . 
more experimental methodologies because they are iacapable of exploring* thought 
processes at other than superficial levels.. ^ Rather, we prefer a more comprehen-A 
sive research strategy in which interviews are used in conjunction with other-/ 
techniques and in which neither flexibility nor rigor are sacrificed. Our goal 
can best be accomplished if, foLlowing protocol analysis, predictions are 



specified and tested on additional data. This .type of follow-up research would 
^provide -additional'^opportunfty for the ^reformulation of hypotheses* and would 

aiso permit mdre educated evaluations by the' research cpmmxmity. Such follow-up 
^ investigations are represented inKpigure T as a "jr&ycling" of the Revised 
. Hypbtjflies is through more traditional relsearch studies, ahd/or ^inatfier series ot 

interviews, ' , • 

As an example^ of research that 'tests Jpredictions derived from protocol 

a'nalysis, we will describe*.a series o£ studies conducted by memberj!^ of our 

> research group who have beeh attempting to. isolate specific diff ic^ulties *that 

^ ■' • . ' * ' . ■ • ■ , . ■ ♦ 

students have in solving algebra word problems .* An example bf the type of 

problem they have been using is^giveri below. 

Write* an equation using the variabiles S and P to represent 
the following statement: ."There are six times as many students 
as professors at this university." Use S for the number of stu- - 
. dents and P for the number of professors. / . 

. This questjion was motivated by the interview segment cited earlier in the 

• ■ . ■ ' • . . . ■ ■ ' ^' ■ ■/ \ 

paper. An initial hypothesis- Vas 'that Mark, and students like him, had diffi-^ 
culty writing equations when the variables involved were abstract, hard to 
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visualize entitities such as -those encountered in, a physics course. However, 
this proved not to be the case. If the above problem is administered, to a 
random group of undergraduates > no more than 60% ajre likely to give the correct 
solution, 6P=S. The most frequently given response isrthe reversed equation, 
6S=P. . ^ •( . 

Approximately 75 indepth interviews have been conducted in which subjects 
have beeti queslkLoned and videotaped as they have attempted to solve .this and 
related problems. The most compelling explanation that has emerged from inter- 
pretive analyses of these interviews is that subjects do not have an operative 
conceptualization of an algebraic equation: they do not view the equation 
6P=S, for example, as involving an operation that is performed on^the ni^ber V 
of professors in order to obtain the number of students. Rather, for many, 
an equation is a passive description of a current state of affairs> According 
to this passive view, the equation 6S=P states that currently there are 6 
students for every pi:ofessor. The equal sign in this case does^ not imply » 
stri^ct equivalency of the expressions on either side^ but singly an association 
between them. / 

Clement,' Lochh^^, and Soloway (1980) tested this hWotheisis by making the 
following prediction: ,that ihe error rate on problems or the type above would 
be decreased i£^ the -problem were placed in a framework £hdt would em^xhasi^e 



the operational nature of an equation. They felt that/ computer pfdgifamming 
provided such a framework. ^ i i!- 



Three experiments were collected in which students with some experi<^nte ii^ 

tten tests cqpjposed ofv. - 



writing computer programs were administered short wri 

■** * -* • ■ 

problems simila'r to the one -above* Students were asked to write or explain 



either i(a) the appropriate algeb^ic equation, or""(b) 
output, the value of one variable when given the value 



a program that would ^ 
of the other. ^-^ In ail - 



EKLC 



,three experimen^ ^tudents^ ma^e fewer etrors using computer language than 
they did with alg^a. On tb^ bas^s of these experiments ,^^l^^nt et al. went 
ott to propose*five specif:^ hypotheses of why the programming context was" suc- 
cessfuj. in reducing errox rat^s. They are currently exploring these hypotheses 
with more indepth interviews- * 

This research exemplifies how the interview can be used in combination 
with other research techniques^;fe<>«;xplore problem solving, at both the indiv-- 
idual and group level. Fijrst^ an interestiiig phenomenon was discovered in a 
informal, tutoriakj^tervL^* Subsequently, more formal indepth interviews 



were used to revise hypotftx^ses through a cycle of hypothesis generation and 
testing* Gi^oi^^^periine|it5 Comparing performance on two types otf problem^ 
served to demonstrate the robustness of the reversal error and IS!^ test the ' 
hypothesis that it resulted from a Passive interpretation qf an eqiia^tibn. Wit 
more interviews being conducted, hypotheses ^iH be cycled through the ptocess 
agaicte As hypotheses become mo r^ refined, related predictions will become , ^ 
more specif ic a^d thus mojjsl^bjectJ^'t^ . 

In the. study by Clement et al*, the hypotfie^^s derived* from the protocols 
was tested in traditional group designs. A strategy which we are just begin-; 
ning t^ adopt in our research involves not only deriving predictions which can 
be JiAd^^i^^^ such group; desi^as, but alsd testing pt:edi\:tions about an indivi-d 
ual's performance during an interview, PrequenttK while we are analyzing a 
protocol, weT^tftfttf^ale hypotheses abt>ut what «the subject is thinking which, in 
turn, suggest probes to test the validity of these hypotheises. In the past, 
these probes haye been u§e4^in the next series of interviews, and their origin 
and^ purpose is rarely coinniunicated iu, the report ' ^ We feel, however, that a 

. > ^ : ' ■ , ■ . - . ■ 

probe so designed constitutes a test of a hypothesis in a traditional sense, 

• . , .■ ■ ■• ► • ^ . . ■ 

and sbfeuld be conveyed as suc>b in tbe research report, this suggests the . - 
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meed- to distinguish between probes designed prior to the' interview to test ^ 
specific ^hjrpotheses and those that are spontaneously developed during the^ W 
interview, - / • 

We ire also beginning to interview subjects on repeated occasions. This 
allows us ^.i^allenge our interpretations of individual protocols, by making 

predictions jDf individual respotises to related problems or additional prqbes- 

V • 
on the same problem. . 

When it Jias been convenient to do so we have asked subjects to comment 

on ^ur analysis of their interview. We have regardea this as an informal^* 

' , ■ . - ^ ■' . ^' ■ • • ' ' a' 

aspect of pur research but are beginning to view this as a method to test and " 

■ . •» ■ , , ' 

develop -"alternative hypotheses. We are presently preparing, to follow up on 

some: interviews concerning student conceptions of pi^obability in which we ? 

\/ill have subjects return a^er we have analyzed an initial intervi^^. ^t 

this time we will test our interpretation by first presenting them with subtly 

dif|Eerent solution s4:rat6g^es anc^ask which they think best characterizes their 

method of solving the problem. UTis'latter technique is limited by the fact 

. ) :■ ' ■ ^ "'^ ' 

that we 'are frequently genaMxing model^ that, elucidate what we think are the 

^tacit levels of a subject's th9ught processes. 'But at the very least we can 

. ■ ■ . \' : ' ■ • ■ ' . ^ - ' 

• ■ |- . ■ ' * • ■■ . ' ■ . ■ ' ■ ■ . ' 

obtain subjects' reffc4ions to the descriptive parts of o\ir analyses. Moreover, 
we are not sureythat* subNectS would be unal^let to recognize tacit aspects of 
their reasoning ^[IXwe could articulate and present thein in an appropriate 
fashion. Eyeri-if tli^ could hot, we are excited at the^prospect of discussing 
with subjects the nature of their thought pYocesises. . For while subjects may 
view themselves as being in ah inferior role during an interview, -we suspect 
that when we approach them5^with our expilanations of their reasoning proce^^s, 
they will feel that they have as much or more insight than we do. They can 
thus providi^ us with alternative hypotheses about how thjey are thinking which .' 
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we can test as ^e do any other hypothesfesV < r . . • 

We. have cpme« to regard the interview not only as a method we.ll suitfed to 
hypothesis generation and testing, but ^Iso as :a common meeting ground, where 
^researchers and participants can perhaps comedo regard each other ?s "cpl- 
leagues engaged in a search. 'V (Raush, 1969, p. 125) 



Reporting the Interview 



to permit 



Thiere are two primary purposes of a scientific report. One is to pe 
other reisearch^rs to replicate, and build, on important work. The second (and 
related)' purpose is to dncourage the scientific community^to evaluate tHe work 
and generate alternative explaaations of the data which may lead, to deeper 
unQerstanding. Driver aad Easiey (1978) made the. corambnt, in reference, to 
Piaget (1974), that it is ^/a pity that the summary of th6 work in Uflderstanding 
"Causality is so brief and reports little t)f -thie actual dialogue with pupils, * 
and that more detailed accounts ^f 3ome of the experiments available have not . 

:^ien translated." "'(p, 76) If it is difficult to include all the information. 

It ' ■ . 

one would want in a bopk-iength report pf'**^n interview study, it becom^^s impos- 
sible to adequately flescribe even a modest endeavor in a journal-lengtl^" 
article. Our suggestions in this section ^ however, are inspired by the lack' 
of attention devoted to what we regard as critical aspects of an interview, 
in cases where limited reporting space could not have been the problem. 

To evaluate the adequacy of interview reporting We reviewed the nine/ ^ 

studies included in Table 1, as well as a study by Karplus (1978). (The 

" ■■ ' ■ ■ ^ ■ ■ . . . ' , ^ ■ ■ . ■ ' ■ c 

Kajrplus study was not included in Table 1 since no information concerning the 

format of the interviews was reported.) Our survey of the literature leaves 

us with the concern that frequently neither of the two purposes stated 



above ife well se'r^ed by reports based on interview research. Certainly with 

interviews it is difficult to co^iinicate clearly.how intfrrpretii^fcns were N 

. ; • >, V . • . ;.: : . •••• .. -.^'^f;, • .. . 

genera teth; from the data. • In many, cases, however, investigators do not, even 

)•'' ^. . ■ N ■ • •,' ■ ' • ' 

report .basic informaliion regardikig the way in which the interview was conducted. 

' ' ■ ' ! ...» 

The following issues ought always to be addressed in the report of inter- 
view research: • ' • ^ - 

Subject Caracteristics > ' * 

Why and how w^tte\;he subjects selected? Ho^ maay were there? What, Were 
their ages, sex, relevant 'educational and socioeconomic characteristics? While 
number and age of subjects is generally reported^ information is bft^n lacking 
aliout how they we ire selected (e..g., Chi et al., 1980; Fir^dette. and Clement, 
1980; Ginsburg, 1977 ; Hebbeler , 1977; KarfeJU*^, 1978; Kennedy, .197.7; Larkin, 
in precis) . * • ^ \ 

Interviewer Char acteristics ' , 

Who did the interviewing and what is ^heir ^experience in^.coQfJucting such 
interviews? What, if any, xelationsfeip prdl/tously existed between^ the inter- v 
viewer an4 the subjeqts?\ While the skill and isxperience of' the int^erviewer . is 
critical, especiallji fpr the indepth and tutorial interviews, we found informa- 
tion about the intervie?^er in a minority of the reports surveyed. In some 

instances the interviewer^as^ ideatified (Davis, 1975; Fredette and plement, 

• ■ « • • ' ' /• . ■ « ^ ' 1- ■' 

1980; Ginsburg, 1977) but the nature of the interviewer ''s prior exp^'tiehce ^and . 

whether or hbt-' there had J)een any ptior experience with the particular subjects 

interviewed w^s virtually never mentioned. . ' . V 

Materials and Instructions 

What problems were given and did subjects read them aloud, silently, or 

were the problems read aloud by the interviewer? What were the .subjects told 

■ . • ■ . . • * "'•»•.■ ■ ■ " ' 

about the purpose of the interview and Jhow were they« instructed to respond? 



Ve fdund that most reports desc^ribed the problems presented, although in some 
cases (e.g., Chi et al., .1980; Davis, 1975; Fredette and Clement^ 1980; 
Karplus, 1978), no inf6rmation was ^iven about the manner in which the problems 

were presented/ We' were somewhat suxrprised to note that instructions to sub- 

" ^ ■ . " • ■ ^ . / ■ . . 

jects were mentioneia- in only a sihall minority of reports- . 

• , ■ * < *^ * **' 

Interview Situation and Characteristics ' • . 

■ .■ - - ■ '\ ■ " • ■ ■ ' ■■ , - : 

^ : What waa\the interview format (thinking-aloud, indepth, ^tutorial)? What * 
types of 3tandardized and spontaneous probes were used? How long was the 

•subject allowed tp r^femain silent before a probe was ^iven? (The method of 
probing is one of the most' important aspects of iritetview research to communi- 
cate accurately, and unless large segments of the. protocols are included in. * 
the report, exampl^ of the typesVand sequencing of probes should be provided. 
How long did the. interview last* and were there problems presented' other than 
the on«s currently (inder discussdJon? If so, how far into the interview wei:e 
th^ current problem^ presented? Were subjects given any training before the 
interview began? What were the physical dharacteristics (e.g., interview loca- 
tion, seating arrangements)? How was the interview recorded and was recording 
equipment in view»of the subject? < ' 

"The interview format was frequently unclear, but could be inferred. The 

' method by which the interview was recorded tills ojnitted in four cases. Each 

of the other characteristics mentioned in this section (practice, other prob- 

* ■ ■■ . * ■ . 

lems, location of problems within interview, ma^^imum silent period ^allowed, 

length and structure of sessions, physical clia:racteristics) were not reported 

by the majority of the research reports we surveyed. 

Analysis -^nd Reporting of Data • ' '. ' 

^ Who ^alyzed the data, and was-analy^sis made primarily from audio or video 

tapes, transcripts, or notes? If the data were jcoded, what was the interrater 



reliability? If descriptive or interpretive analyses were tnade,, how were they 
conducted? For example , was the analysis performed by a single 'individual , oi^ 
were several involved and did they work separately or as a group? |Iow were 

IV 

, ■ . f \ • 

disagreements among individuals analyzing' the data negotiated? If interpreta- 
tions or conclusions are illustrated by excerpts from subject protocols, hpw 
represlrftative are these excerpts of subjects in general? How many subjects 
seem^to fit^^interpretations given, and what statements remain poorly accounted 

■ ' 

for? How did subjects react to the interview? Were thay nervous, resi^stant, 

r • . .. 

distracted, etjc.? \^ . - - 

While^^t^^-Jrevel of analysis (coded, descriptive or interpretive) could 
generally be inferred, not^all studies employing coded analysis* reporteid ' ^ \ \ 
any^ measure of interrater reliabilifty (e.^g., HeyBbelei;, 1977; Karjplus, 1978) nor 
did they always indicate whether more thaq one rater was involved. Only two 
studies reviewed used an interpretive analysis, aiid pne of these (Fredette and 
Clement, 1980): failed to indicate who had participated in the analysis. It 
was not always clear how representative repotted intervievj segments were of 
the other interviews (e.g., Chi et al., 1980; Ginsburg, 1,977) . 

If our suggestions seem simplistic, one, need read only a' few research 

■ . : \ ^ ■ . 

teports to appteciat^e the laxity .with which the interview study is often, 
reported.. Be caps^^^^f^the nature of the interview, the research community is^ 
j)laced in the positi'an of having access to reports'with little of the origi-' 
nal data available, {^e only mention we. came across concerning theyavail-' 
ability of transcripts was^ a parenthetical comment in Davis (1973^ which pro- 

mised a complete transcript to "anyone who wants it badly enough," p. 38.) 

^ _ ^ ■ ^ . ^ - 

Given such limited access to data, it is even more important for investigators 

' ■ . I * • ■' ' \' , ' . . *' ■ . ' - ■ ■ ' 

to rep'ort as clearly as possible the details of how the interview was con- 

•■ ■ ■ . . 

ducted aq^ how the data were analyzed. It^would also be highly desirable, t 



make transcripts or^ tapes available uppn request* This would facilitate the 
diff ici^t task of evaluating interview Research and should encourage f dllow-up 
work by^ther research groups. ^ ' . ^ 

c " * ' / . • ■ ' • , " 

. ^ Conclusions * . 

■ • ~. ■ " • • 

. r ■ . ■ ' • . ' ■ • ■ " ' ; ' . ' ■ , • •■; 

^' . ■ . ! < . . ^ \ 

We Jw^ve characterized, the interview not only as an informal ^oiirce of 

bypotlies^s but^as a' general method for hypothesis generation and test;ii^. , /We 

. 'J- ■ " • .... * * ' ' ' '■ " 

have implied that the intescview iiDiay be a more powerful method of investigating 

certain aspects of ^cognition than more traditional, experimental methods « H6w- 

(BV^r, we feel that t4ie effort to understand cognitive pl^iomena- fs ill-served [ 

byjrestricting the methodologies that are used, and have ^tgued for a research 

afipToach in which the interview i^ used in concert with traditional experimential 

design- ^ ^ 

Because of the flexibility of the indepth interview and, the 4lif fi^culty 

of conducting and reporting descriptive and inte^rpretiye analyses , :^e have 

' ■ , ' • # fl 

. ■ ■ " ■' 

drgued that the research report should provide detailed information about liow ^ 
the interview proceeded and how the data were analyzed, and that interview 
transcripts should be made^available .to interested parties, Ve have emphasized 
the need to adopt a critical' orientation toward explanations derived from inter- 
view^ data. While our suggestions are informal and simplistic, we are confident 

. ■■ ^ -.. . ^ 

that further dfscussion of the issues to which they were directed will result in 
refinements »of the* interview methodology. 
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Type of Interview 

; ... . ' -1 ' ■ ' ■ . V ' w. - . 

TiTjnking-aloud . Indepth Tutorial 





Coded' 


Chi et al., 1980 
Larkin, in press 


;Hebbeler, 1977 
Russell, 1977 




Levels 








Davis, 1975a ? 










Kennedy, 1977 . 




Descriptive 






Rbsnick and • 


Analysis 








Clement, in press 


y 


Inte»tpretive 




Fredette'and 
Clement, 1980 

Ginsburg, 1977 


• • • 



Table 1. Studies categorized with respect to type of interviev/ and 
level of analysis. 
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(Interview) 



; Questionnaires 
Naturalistic Observations 

Controlled Laboratory Study 
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■Revised 
Hypothesis 



.1 



Figure 1. The interview as a component of a gerf 
formulating and testing .hypotheses. 
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